MULTILINGUAL MODEL

Mitigating Language-Dependent Ethnic Bias in BERT (2021.emnlp-main)

Challenge:	a multilingual model is not specifically tailored to either the language nor the application domain.
Approach:	They propose a CRF-based baseline approach and multilingual BERT to the task . they achieve an F-score of 88% on the development data and 87% on the test set with BERT .
Outcome:	The proposed model achieves an F score of 88% on the development data and 87% on the test set with BERT.

Copied to clipboard

Jaimeen Ahn, Alice Oh

Gradient-based Gradual Pruning for Language-Specific Multilingual Neural Machine Translation (2023.emnlp-main)

Challenge:	Ethnic bias is one of the most prevalent social stereotypes.
Approach:	They propose to use a multilingual model and contextual word alignment to mitigate ethnic bias in monolingual BERT for English, German, Spanish, Korean, Turkish, and Chinese.
Outcome:	The proposed methods alleviate ethnic bias in English, German, Spanish, Korean, Turkish, and Chinese using a multilingual model and contextual word alignment of two monolingual models.

Copied to clipboard

Dan He, Minh-Quang Pham, Thanh-Le Ha, Marco Turchi

What Language Model to Train if You Have One Million GPU Hours? (2022.findings-emnlp)

Challenge:	Multilingual neural machine translation suffers from performance degradation in high-resource languages compared to bilingual counterparts.
Approach:	They propose a gradient-based gradual pruning technique for multilingual neural machine translation that allows for partial parameter sharing across language pairs to alleviate interference.
Outcome:	The proposed approach yields a notable performance gain on IWSLT and WMT datasets.

Copied to clipboard

Teven Le Scao, Thomas Wang, Daniel Hesslow, Stas Bekman, M Saiful Bari, Stella Biderman, Hady Elsahar, Niklas Muennighoff, Jason Phang, Ofir Press, Colin Raffel, Victor Sanh, Sheng Shen, Lintang Sutawika, Jaesung Tae, Zheng Xin Yong, Julien Launay, Iz Beltagy

Cross-Lingual Transfer from Related Languages: Treating Low-Resource Maltese as Multilingual Code-Switching (2024.eacl-long)

Challenge:	Recent years have seen the advent of large language models characterized by emergent capabilities arising from sheer scale alone.
Approach:	They propose to use a multilingual model to compare performance to the English-only model by ablation at the billion-parameter scale.
Outcome:	The proposed model is based on a multilingual model and its performance against the English-only model.

Copied to clipboard

Kurt Micallef, Nizar Habash, Claudia Borg, Fadhl Eryani, Houda Bouamor

No Error Left Behind: Multilingual Grammatical Error Correction with Pre-trained Translation Models (2024.eacl-long)

Challenge:	Multilingual models exhibit impressive cross-lingual transfer capabilities on unseen languages, but performance is impacted when there is a script disparity with the languages used in the model’s pre-training data.
Approach:	They propose a novel method to align a resource-rich language's script with a target language and train a classifier that can make informed decisions regarding the appropriate processing of each token.
Outcome:	The proposed model can be used to transfer a language's scripts across multiple languages, but it is suboptimal for mixed languages, where only a subset benefits while the rest is impeded.

Copied to clipboard

Agnes Luhtaru, Elizaveta Korotkova, Mark Fishel

A Multi-layered Approach to Physical Commonsense Understanding: Creation and Evaluation of an Italian Dataset (2024.lrec-main)

Challenge:	Grammatical Error Correction (GEC) research has primarily focused on English with little coverage for other languages.
Approach:	They propose a multilingual machine translation model that can be fine-tuned to improve error correction out-of-the-box.
Outcome:	The proposed model outperforms similar-sized MT5 models and competes favourably with larger models.

Copied to clipboard

Giulia Pensa, Begoña Altuna, Itziar Gonzalez-Dios

Reasoning

Challenge:	Using a multilingual model, we examine the ability of large language models to perform reasoning tasks.
Approach:	They propose to use a multilingual model to analyze commonsense reasoning in large language models for Italian and to provide a semi-automated system to complete the annotation.
Outcome:	The proposed model performs at high-level classification tasks but its easoning is inconsistent and unverifiable, since it does not capture intermediate evidence.

Discovering Representation Sprachbund For Multilingual Pre-Training (2021.findings-emnlp)

Copied to clipboard

Yimin Fan, Yaobo Liang, Alexandre Muzio, Hany Hassan, Houqiang Li, Ming Zhou, Nan Duan

Multilingual Neural Machine Translation with Language Clustering (D19-1)

Challenge:	Existing models perform poorly on many languages and cross-lingual tasks due to typological differences and contradictions between some languages.
Approach:	They propose to pre-train multilingual pre-trained models to handle cross-lingual tasks in one model.
Outcome:	The proposed model improves performance on cross-lingual tasks compared to baselines on multiple languages .

Copied to clipboard

Xu Tan, Jiale Chen, Di He, Yingce Xia, Tao Qin, Tie-Yan Liu

A Unified Framework for Multilingual and Code-Mixed Visual Question Answering (2020.aacl-main)

Challenge:	Existing work on multilingual neural machine translation has been neglected due to its burdensome training process.
Approach:	They develop a framework that clusters languages into different groups and trains one multilingual model for each cluster.
Outcome:	The proposed model reduces the cost of training and improves translation accuracy.

Copied to clipboard

Deepak Gupta, Pabitra Lenka, Asif Ekbal, Pushpak Bhattacharyya

Multimodality

Challenge:	Existing techniques for visual question answering focus on English questions, but many applications require a multilingual module.
Approach:	They propose a deep learning framework for multilingual and code- mixed visual question answering . they create Hindi and Code-mixed VQA datasets by exploiting linguistic properties of these languages .
Outcome:	The proposed model is capable of predicting answers from the questions in Hindi, English or Code- mixed (Hindi-English) languages.

Assessing Reference-Free Peer Evaluation for Machine Translation (2021.naacl-main)

Copied to clipboard

Sweta Agrawal, George Foster, Markus Freitag, Colin Cherry

Condensing Multilingual Knowledge with Lightweight Language-Specific Modules (2023.emnlp-main)

Challenge:	Existing methods to evaluate machine translation output are based on comparing MT output to one or more reference translations.
Approach:	They propose to use probabilities given by a large, multilingual model as a reference-free metric.
Outcome:	The proposed model is robust and likely to offer reasonable performance across a broad spectrum of domains and different system qualities.

Copied to clipboard

Haoran Xu, Weiting Tan, Shuyue Li, Yunmo Chen, Benjamin Van Durme, Philipp Koehn, Kenton Murray

Green & Sustainable NLP

Challenge:	Existing methods to boost performance in multilingual models but scalability is difficult to manage.
Approach:	They propose a method that incorporates language-specific (LS) modules to boost model performance.
Outcome:	The proposed method outperforms state-of-the-art methods while outperforming existing methods.

Learning to Solve NLP Tasks in an Incremental Number of Languages (2021.acl-short)

Copied to clipboard

Giuseppe Castellucci, Simone Filice, Danilo Croce, Roberto Basili

Challenge:	Existing models that support multiple languages are expensive and time-consuming to retrain . retraining only with the new language data will result in Catastrophic Forgetting of previously acquired knowledge.
Approach:	They propose a Continual Learning strategy that updates a model to support new languages over time while maintaining consistent results on previously learned languages.
Outcome:	The proposed model is based on an existing model that "teaches" to a student while the student is also trained on a new language.

Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies (P19-1)

Copied to clipboard

Yunsu Kim, Yingbo Gao, Hermann Ney

Language-aware Interlingua for Multilingual Neural Machine Translation (2020.acl-main)

Challenge:	Existing approaches to transfer a pretrained NMT model to a new, unrelated language without shared vocabularies are limited to cognate languages.
Approach:	They propose to transfer a pretrained NMT model to a new, unrelated language without shared vocabularies by using cross-lingual word embedding and injecting artificial noises.
Outcome:	The proposed methods outperform multilingual joint training by a large margin in five low-resource translation tasks.

Copied to clipboard

Changfeng Zhu, Heng Yu, Shanbo Cheng, Weihua Luo

COVID-19 Named Entity Recognition for Vietnamese (2021.naacl-main)

Challenge:	Existing multilingual neural machine translation models fail to capture diversity and specificity of different languages, resulting in inferior performance against individual models that are sufficiently trained.
Approach:	They propose to integrate a language-aware interlingua into an Encoder-Decoder architecture to learn a semantic representation from the semantic spaces of different languages while allowing for language-specific specialization of a particular language pair.
Outcome:	The proposed model achieves remarkable improvements over state-of-the-art multilingual NMT models and produces comparable performance with strong individual models.

Copied to clipboard

Thinh Hung Truong, Mai Hoang Dao, Dat Quoc Nguyen

Information Extraction & Text Mining

Challenge:	a new dataset is being developed to help fight the COVID-19 pandemic . the dataset is annotated for the named entity recognition task with newly-defined entity types .
Approach:	They present the first manually-annotated COVID-19 domain-specific dataset for Vietnamese . their dataset is annotated for the named entity recognition task with newly-defined entity types .
Outcome:	The proposed dataset is the first manually-annotated COVID-19 domain-specific dataset for Vietnamese.

Comparing Explanation Faithfulness between Multilingual and Monolingual Fine-tuned Language Models (2024.naacl-long)

Copied to clipboard

Zhixue Zhao, Nikolaos Aletras

Explainability & Interpretability in NLP

Challenge:	Previous studies have investigated how different factors affect faithfulness of model explanations .
Approach:	They find that the larger the multilingual model, the less faithful FAs are compared to its counterpart monolingual models.
Outcome:	The results show that the larger the multilingual model, the less faithful the FAs are compared to its counterpart monolingual models.

mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus (2025.findings-acl)

Copied to clipboard

Matthieu Futeral, Armel Randy Zebaze, Pedro Ortiz Suarez, Julien Abadji, Rémi Lacroix, Cordelia Schmid, Rachel Bawden, Benoît Sagot

Multimodality

Challenge:	Existing studies show that multimodal large language models can learn from text-image data.
Approach:	They propose to train multimodal large language models on large amounts of text-image data . they also show a boost in few-shot learning performance across various multilingual tasks .
Outcome:	The proposed dataset is not public and is only in English . it is the first large-scale multilingual and multimodal document corpus crawled from the web.

Counter-Interference Adapter for Multilingual Machine Translation (2021.findings-emnlp)

Copied to clipboard

Yaoming Zhu, Jiangtao Feng, Chengqi Zhao, Mingxuan Wang, Lei Li

How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models (2021.acl-long)

Challenge:	Existing approaches to multilingual machine translation suffer from performance degradation, resulting in a single model being inferior to separately trained bilingual models on resource-rich languages.
Approach:	They propose a transformer-based model with a small parameter overhead for multilingual machine translation that outperforms strong multilingual baselines on 64 of 66 language directions.
Outcome:	The proposed model outperforms strong multilingual baselines on 64 of 66 language directions, 42 of which have above 0.5 BLEU improvement.

Copied to clipboard

Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, Iryna Gurevych

Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation (2020.coling-main)

Challenge:	Using pretraining data, we find that a designated monolingual tokenizer plays an equally important role in the downstream performance of the model.
Approach:	They propose to compare pretrained multilingual models with their monolingual counterparts on a set of five diverse monolingual downstream tasks.
Outcome:	The proposed models offer previously unmatched performance in all NLP tasks.

Copied to clipboard

Junhao Liu, Linjun Shou, Jian Pei, Ming Gong, Min Yang, Daxin Jiang

Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models (2021.eacl-main)

Challenge:	Cross-lingual Machine Reading Comprehension (CLMRC) is a challenging problem due to the lack of large-scale annotated datasets in low-source languages, such as Arabic, Hindi, and Vietnamese.
Approach:	They propose a novel approach to augment cross-lingual machine reading comprehension by combining knowledge from multiple language branch models into a single model for all target languages.
Outcome:	Extensive experiments on two CLMRC benchmarks show the proposed method is effective and robust to data noises.

Copied to clipboard

Nora Kassner, Philipp Dufter, Hinrich Schütze

mDAPT: Multilingual Domain Adaptive Pretraining in a Single Model (2021.findings-emnlp)

Challenge:	Recent work has shown that monolingual English language models can be used as knowledge bases.
Approach:	They use mBERT to query monolingual English language models with masked sentences to test their hypothesis.
Outcome:	The proposed model can be used to answer fill-in-the-blank questions in 53 languages.

Copied to clipboard

Rasmus Kær Jørgensen, Mareike Hartmann, Xiang Dai, Desmond Elliott

Exploring Multilingual Pre-trained Language Model for Aspect-based Sentiment Analysis (2026.findings-acl)

Challenge:	Existing domain-specific multilingual pretraining data is difficult to obtain due to regulations, legislation, or simply a lack of language- and domain- specific text.
Approach:	They propose to continue pretraining a language model on domain-specific unlabelled text . this allows for better modelling of text for downstream tasks within the domain .
Outcome:	The proposed approach outperforms the general multilingual model and performs close to its monolingual counterpart.

Copied to clipboard

Ye Wang, Ruijun Jiang, Zhongqing Wang, Guodong Zhou

Structure-Level Knowledge Distillation For Multilingual Sequence Labeling (2020.acl-main)

Challenge:	Aspect-based sentiment analysis studies have focused on English datasets, but labeled data is scarce.
Approach:	They propose a multilingual pre-trained language model that leverages bilingual pre-training to leverage aspects-based sentiment analysis.
Outcome:	The proposed model outperforms state-of-the-art models across multiple languages.

Copied to clipboard

Xinyu Wang, Yong Jiang, Nguyen Bach, Tao Wang, Fei Huang, Kewei Tu

An Open Multilingual System for Scoring Readability of Wikipedia (2024.acl-long)

Challenge:	Existing multilingual models still underperform individual monolingual models due to model capacity limitations.
Approach:	They propose to distill the structural knowledge of several monolingual models (teachers) to the unified multilingual model (student).
Outcome:	The proposed model outperforms strong baseline models and teacher models on 4 multilingual tasks with 25 datasets and has stronger zero-shot generalizability.

Copied to clipboard

Mykola Trokhymovych, Indira Sen, Martin Gerlach

CINO: A Chinese Minority Pre-trained Language Model (2022.coling-1)

Challenge:	Existing studies on the readability of Wikipedia have focused on English only and there are currently no systems supporting automatic readability assessment of the 300+ languages in Wikipedia.
Approach:	They propose a multilingual model to assess Wikipedia's readability using a dataset spanning 14 languages.
Outcome:	The proposed model outperforms existing models in a zero-shot scenario and is more accurate than previous benchmarks.

Copied to clipboard

Ziqing Yang, Zihang Xu, Yiming Cui, Baoxin Wang, Min Lin, Dayong Wu, Zhigang Chen

Multilingual unsupervised sequence segmentation transfers to extremely low-resource languages (2022.acl-long)

Challenge:	Existing multilingual pre-trained language models do not perform well on some low-resource languages.
Approach:	They propose a multilingual pre-trained language model for Chinese minority languages . they collect documents from Wikipedia and construct two classification datasets .
Outcome:	The proposed model outperforms baseline models on various classification tasks.

Copied to clipboard

C. Downey, Shannon Drizin, Levon Haroutunian, Shivin Thukral

Examining Cross-lingual Contextual Embeddings with Orthogonal Structural Probes (2021.emnlp-main)

Challenge:	Unsupervised sequence segmentation is a key component of low-resource languages where there is little or no gold-standard data on which to train supervised models.
Approach:	They propose to pre-train a Masked Segmental Language Model multilingually to achieve unsupervised segmentation performance in extremely low-resource languages.
Outcome:	The proposed model outperforms a monolingual model and a pre-trained model on Quechua in 6/10 settings.

Copied to clipboard

Tomasz Limisiewicz, David Mareček

Zero-shot Disfluency Detection for Indian Languages (2022.coling-1)

Challenge:	Existing studies on whether multilingual embeddings can be aligned in a shared space across languages are lacking.
Approach:	They propose to learn a projection based on monolingual annotated datasets and evaluate syntactic and lexical information encoded in a shared cross-lingual embedding space.
Outcome:	The proposed model can be used to learn representations for languages with low resources.

Copied to clipboard

Rohit Kundu, Preethi Jyothi, Pushpak Bhattacharyya

Revisiting Machine Translation for Cross-lingual Classification (2023.emnlp-main)

Challenge:	Disfluency correction models can help alleviate this problem, but the unavailability of labeled data in low-resource languages impairs progress.
Approach:	They propose to use a pretrained multilingual model to detect zero-shot disfluency in Indian languages.
Outcome:	The proposed model achieves F1 scores of 75 and higher on five disfluency types across four languages.

Copied to clipboard

Mikel Artetxe, Vedanuj Goswami, Shruti Bhosale, Angela Fan, Luke Zettlemoyer

Multilingual Simultaneous Neural Machine Translation (2021.findings-acl)

Challenge:	Recent work in cross-lingual learning has pivoted around multilingual models, which are typically pretrained on unlabeled corpora in multiple languages using some form of language modeling objective.
Approach:	They propose to use a stronger machine translation system to mitigat mismatch between training on original text and running inference on machine translated text.
Outcome:	The proposed approach is highly task dependent and calls into question the dominance of multilingual models for cross-lingual classification.

Copied to clipboard

Philip Arthur, Dongwon Ryu, Gholamreza Haffari

Jamo Pair Encoding: Subcharacter Representation-based Extreme Korean Vocabulary Compression for Efficient Subword Tokenization (2020.lrec-1)

Challenge:	Simultaneous machine translation (SIMT) involves translating source utterances to the target language in real-time before the speaker utterrance completes.
Approach:	They propose a multilingual approach to simultaneous machine translation where a single model simultaneously translates between multiple languages.
Outcome:	The proposed multilingual approach improves on two Germanic and three Romance languages and is on-par or better than the universal model trained for all languages.

Copied to clipboard

Sangwhan Moon, Naoaki Okazaki

Vocabulary-informed Language Encoding (2022.coling-1)

Challenge:	a vocabulary size problem is unsolved in multilingual pre-training tasks.
Approach:	They propose two algorithms that increase the elasticity of budget for building the vocabulary in Byte-Pair Encoding inspired tokenizers, and reduce the cost of supporting Korean in a multilingual model.
Outcome:	The proposed method reduces the cost of supporting Korean in a multilingual model.

Copied to clipboard

Xi Ai, Bin Fang

Adapting Monolingual Models: Data can be Scarce when Language Similarity is High (2021.findings-acl)

Challenge:	A Multilingual model relies on language encodings to identify input languages . a method to compute a vocabulary-informed language coding can improve multilingual models .
Approach:	They propose a method to compute a vocabulary-informed language encoding as the language representation for a required language.
Outcome:	The proposed method improves performance on unsupervised translation and cross-lingual embedding.

Copied to clipboard

Wietse de Vries, Martijn Bartelds, Malvina Nissim, Martijn Wieling

The Thieves on Sesame Street are Polyglots - Extracting Multilingual Models from Monolingual APIs (2020.emnlp-main)

Challenge:	Large pre-trained language models are the dominant approach for solving many tasks in natural language processing.
Approach:	They propose to retrain the lexical layers of four BERT-based models using data from two low-resource target languages while the Transformer layers are independently finetuned on a POS-tagging task in the model's source language.
Outcome:	The proposed method achieves high performance for both target and target languages with high similarity.

Copied to clipboard

Nitish Shirish Keskar, Bryan McCann, Caiming Xiong, Richard Socher

Efficient Multilingual Multi-modal Pre-training through Triple Contrastive Loss (2022.coling-1)

Challenge:	Recent work has demonstrated that deployed NLP models can be stolen by adversaries by querying victim models with gibberish input data that consists of random sequences of words.
Approach:	They propose to extract a local copy of a monolingual victim model from an API and query it with gibberish input data paired with the victim's labels.
Outcome:	The extracted model learns the task from the monolingual victim, but it generalizes far better than the victim to several other languages.

Copied to clipboard

Youhan Lee, KyungTae Lim, Woonhyuk Baek, Byungseok Roh, Saehoon Kim

A Survey of Multilingual Models for Automatic Speech Recognition (2022.lrec-1)

Challenge:	Existing approaches to learn visual and textual representations from web-scale image-text pairs are limited due to labeling cost and limited scalability.
Approach:	They propose to use web-scale image-text pairs to learn visual and textual representations in the shared space.
Outcome:	The proposed enhancement scheme improves multilingual vision-and-language tasks by minimizing a triplet contrastive loss on images and two different language texts with the same meaning.

Copied to clipboard

Hemant Yadav, Sunayana Sitaram

Speech & Audio in NLP

Challenge:	Automatic Speech Recognition (ASR) systems have achieved human-like performance for a few languages, but the majority of the world’s languages do not have usable systems due to the lack of large speech datasets to train these models.
Approach:	They propose to use unlabeled speech data to build multilingual ASR models that can be used for improved performance on low-resource languages.
Outcome:	The proposed models can be used to improve performance on low-resource languages by using unlabeled speech data.

MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer (2020.emnlp-main)

Copied to clipboard

Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, Sebastian Ruder

Parameter-Efficient Finetuning for Robust Continual Multilingual Learning (2023.findings-acl)

Challenge:	Current deep pretrained models lack capacity to represent all languages . limited capacity is an issue even for high-resource languages where models are not included in training data at all.
Approach:	They propose an adapter-based framework that enables high portability and parameter-efficient transfer to arbitrary tasks and languages by learning modular language and task representations.
Outcome:	The proposed framework outperforms state-of-the-art models on cross-lingual transfer across languages and typologically diverse models.

Copied to clipboard

Kartikeya Badola, Shachi Dave, Partha Talukdar

A Robust Self-Learning Framework for Cross-Lingual Text Classification (D19-1)

Challenge:	Existing approaches to Continual Multilingual Learning (CML) are based on updating models using new data in stages.
Approach:	They propose a parameter-efficient finetuning strategy to increase the number of languages on which the model improves after an update while reducing the magnitude of loss for the remaining languages.
Outcome:	The proposed model improves on the languages included in the latest update while reducing the loss of performance on the remaining languages.

Copied to clipboard

Xin Dong, Gerard de Melo

TL-CL: Task And Language Incremental Continual Learning (2024.emnlp-main)

Challenge:	Recent advances in pretrained contextual representation models have made significant progress on a number of different English NLP tasks.
Approach:	They propose a robust framework to include unlabeled non-English samples in the fine-tuning process of pretrained multilingual representation models.
Outcome:	The proposed framework includes unlabeled non-English samples in the fine-tuning process of pretrained multilingual representation models.

Copied to clipboard

Shrey Satapara, P. K. Srijith

Green & Sustainable NLP

Challenge:	a multilingual model is periodically updated to accommodate new tasks in previously learned languages or new languages for established tasks.
Approach:	They propose an adapter-based parameter-efficient fine-tuning strategy for continual learning in multilingual models.
Outcome:	The proposed approach outperforms other parameter-efficient approaches without access to historical data for replay.

Adaptive Token-level Cross-lingual Feature Mixing for Multilingual Neural Machine Translation (2022.emnlp-main)

Copied to clipboard

Junpeng Liu, Kaiyu Huang, Jiuyi Li, Huan Liu, Jinsong Su, Degen Huang

Translationese as a Language in “Multilingual” NMT (2020.acl-main)

Challenge:	Multilingual neural machine translation models can translate multiple language pairs in a single model but lacks ability to capture language-specific features.
Approach:	They propose a token-level feature mixing method that captures different features and dynamically determines feature sharing across languages.
Outcome:	The proposed method outperforms baselines and can be extended to zero-shot translation.

Copied to clipboard

Parker Riley, Isaac Caswell, Markus Freitag, David Grangier

IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation (2021.emnlp-main)

Challenge:	Recent work examines the impact of translationese in machine translation evaluation using the WMT evaluation campaign.
Approach:	They propose to use a sentence-level classifier to distinguish translationese from original target text to generate a machine translation model that can produce more natural outputs at test time.
Outcome:	The proposed model produces more natural outputs at test time, yielding gains in human evaluation scores on accuracy and fluency.

Copied to clipboard

Samuel Cahyawijaya, Genta Indra Winata, Bryan Wilie, Karissa Vincentio, Xiaohong Li, Adhiguna Kuncoro, Sebastian Ruder, Zhi Yuan Lim, Syafri Bahar, Masayu Khodra, Ayu Purwarianti, Pascale Fung

Improving Cross-lingual Transfer with Contrastive Negative Learning and Self-training (2024.lrec-main)

Challenge:	Lack of publicly available NLG benchmarks for low-resource languages poses a challenge . authors show that IndoBART and IndoGPT achieve competitive performance on all tasks .
Approach:	They propose a benchmark to measure natural language generation progress in three low-resource languages of Indonesia . they use a corpus of pretraining datasets to build their models .
Outcome:	The proposed benchmark measures progress in Indonesian, Javanese, and Sundanese . the results highlight the importance of pretraining on closely related, localized languages .

Copied to clipboard

Guanlin Li, Xuechen Zhao, Amir Jafari, Wenhao Shao, Reza Farahbakhsh, Noel Crespi

Cross-Lingual Transfer

Challenge:	Recent studies improve cross-lingual transfer learning by better aligning the internal representations within the multilingual model or exploring the information of the target language using self-training.
Approach:	They propose to use negative pairs to align the multilingual model and self-train the model to converge on the obtained clean pseudo-labels.
Outcome:	The proposed method improves upon the baseline models and can serve as a beneficial complement to the alignment-based methods.

Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution (2023.findings-acl)

Copied to clipboard

Tianjian Li, Kenton Murray

Multilingual Transfer Learning for Children Automatic Speech Recognition (2022.lrec-1)

Challenge:	Existing studies on cross-lingual transferability of multilingual LMs show that they can perform tasks in low-resource languages.
Approach:	They propose a method to regularize the model from learning language invariant representations and a way to select model checkpoints without a development set in the target language.
Outcome:	The proposed method reduces the accidental translation problem by 68% and improves the ROUGE-L score by 1.5 on average.

Copied to clipboard

Thomas Rolland, Alberto Abad, Catia Cucchiarini, Helmer Strik

Multimodality

Challenge:	Recent advances in automatic speech recognition (ASR) systems have been criticized for high acoustic variability and limited amount of available training data.
Approach:	They propose a two-step training strategy that uses multilingual learning followed by language-specific transfer learning to generalize children's speech.
Outcome:	The proposed training strategy outperforms single language training and multilingual and transfer learning alone in English.

An LLM-based Temporal-spatial Data Generation and Fusion Approach for Early Detection of Late Onset Alzheimer’s Disease (LOAD) Stagings Especially in Chinese and English-speaking Populations (2025.findings-emnlp)

Copied to clipboard

Yang Han, Jacqueline C.k. Lam, Victor O.k. Li, Lawrence Y. L. Cheung